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Program evaluation has long been a useful technical tool for deternnining if programs 
are meeting their stated goals. Specialists submit reports that help administrators to 
decide changes in curriculum content or direction. 

In recent years program evaluators have taken on an expanded role because their 
experience can be of value in every stage of the development of the program. This 
Digest introduces the reader to the scope of evaluation and the changing roles 
evaluators are asked to play in the school district. 

HOW ARE EDUCATIONAL PROGRAMS 
EVALUATED? 

Every area of school curriculum is designed with certain goals in mind. A program 
evaluation measures the outcome of a program based on its student-attainment goals, 
level of implementation, and external factors such as budgetary constraints and 
community support. 

Three categories of instructional program evaluation are described by Bruce Wayne 
Tuckman (1985). "Formative evaluation" is an internal function that feeds results back 
into the program to improve an existing educational unit; this kind of evaluation is used 
frequently by teachers and school administrators to compare outcomes with goals. 
Attainment can be measured and procedures modified over time. 

"Summative evaluation" exists for the purpose of demonstration and documentation. 
Various ways of achieving similar goals can be compared. Summative evaluations help 
school districts analyze their unique characteristics and choose the program that will 
best achieve their pedagogical goals. An example is the evaluation of the adaptability 
and success in the work force of students who have emerged from a program. 

"Ex post facto evaluation" is a study over time. It attempts to determine if new programs, 
launched without readily predictable results, are achieving the desired goals. Here the 
data generated by continuous analysis are compared over time and, when available, 
compared with data of similar pilot programs. Both longitudinal (comparison of results 
over time) and cross-sectional (comparison of different student groups) results give 
evaluators the data to recommend improvement or termination. 

HOW HAVE NONTRADITIONAL 
MEASUREMENTS AFFECTED PROGRAM 
EVALUATION? 

The first and most important issue in evaluation-how well students achieve mastery of 
new facts and skills-can often be measured by standardized tests. Verifications of 
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reliability and validity are the litmus tests of these standardized evaluation tools. 
Reliability is the achievement of consistency in results. Consistency is measured in 
several ways: by comparing test results over time (giving the same test at intervals), by 
grade level expectations, and by national percentile rankings. Validity is the degree to 
which a test actually measures what it claims to measure, that is, the successful 
appropriation of intended subject matter. 

However, standardized testing involves a plethora of statistical uncertainties that have 
led some program evaluators to adopt other techniques to measure student attainment. 
Several alternative testing methods are being used: (1) standardized interviews allow 
students' responses to be compared and summarized; (2) direct tests (sometimes 
verbal) such as reading and math demonstrations enable teachers to gauge strengths 
and weaknesses and determine competency beyond mere right and wrong answers; 
and (3) students' notes, art work, and other material can be inspected for evidence of 
mastery. 

Edward F. DeRoach (1988) thinks that relying on an array of achievement, literacy, and 
minimal competency testing overemphasizes cognitive-achievement factors while 
disregarding affective-aesthetic development. He suggests using a program evaluation 
profile that reveals less tangible values such as: (1) program description that evaluates 
the nature of the community and the cultural/occupational background of parents; (2) 
program objectives that would measure performance in American history, for example, 
by involvement in school political activity or community service; (3) program content that 
ranges from knowledge of the facts to facility with placing information in larger contexts; 
and (4) processes that measure listening, questioning, summarizing, solving, and 
creating skills, as well as social skills such as tolerance, respect, and fairness to others. 
It remains unclear whether such "performance-based" assessments can be usefully 
compared across wide-ranging student populations. 

HOW DOES COMMUNITY AND SCHOOL BOARD 
INPUT AFFECT PROGRAM EVALUATION? 

The role of citizen judgments in program evaluation was the focus of four studies 
conducted by the Northwest Regional Educational Laboratory in Portland, Oregon. Nick 
L. Smith (1983) notes the growing pressure for citizens and their representatives 
(school boards) to participate in school planning and review activities. Based on the 
American tradition of local control of education, it is thought that increased parental 
participation on boards developing new educational philosophies and innovative 
curricula would make school district programs more responsive to local ideological, 
economic, and cultural values. 

The study concluded that citizen judgments must be used judiciously to avoid bias, but 
that such judgments can be predictive of community responsiveness and receptivity to 
future collaboration. Program evaluators have paid more attention to political factors in 
recent years as evaluation has become a stronger force in program design. Hence, 
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attention to public sentiment needs to be a high priority. 

HOW DO ADMINISTRATORS VIEW 
EVALUATIONS? 

For principals and superintendents, the purpose of program evaluation is to provide 
information to help them make decisions regarding programs. In general, principals feel 
that the benefits of evaluations are minimal because of their inability to measure 
program components that are of real importance, or because principals' own proximity 
to the everyday realities of the educational process gives them what they feel is a better 
basis for understanding needs and implementing change. Superintendents tend to be 
more positive about the value of program evaluation. In particular, evaluations that 
reported deficiencies and discussed possible solutions were highly rated. Second in 
importance are personal meetings with evaluation personnel. 
In small schools, the missing element in evaluations seems to be the attempt to make 
such studies systematic, purposive, cyclical, comprehensive, and well-communicated 
(James R. Sanders 1988). Sanders suggests that a Program Review Committee (PRC), 
composed of the superintendent, principal, grade level chairperson, and an educational 
specialist, be established. Each year the committee should conduct a review of one or 
two programs, so that each program receives careful scrutiny once every five years. 

WHAT ARE THE NEW ROLES FOR 
E VALUATORS? 



According to Jody L. Fitzpatrick (1988), the job of the evaluator is expanding from 
technical roles to political and advisory roles. In innovative programs, defined as those 
still in a research and development phase, evaluators help identify goals and develop a 
strategy for accomplishing these goals. 

Another new role for the evaluator is translating policy questions developed by school 
boards and legislators into the more precise questions of program evaluation. In this 
role, the evaluator helps fashion new and innovative programs with features that are 
readily measurable. Once pilot programs are begun, the evaluator then has the 
opportunity to determine how fully the program was implemented before evaluating its 
effectiveness. According to Fitzpatrick, evaluation questions imply certain design 
decisions. Besides content, these questions can help determine the parameters of cost, 
time, and the availability of professional personnel. 

The program manager can monitor the innovative program through the oral briefings 
and written reports of the program evaluator. To be effective, communication should be 
ongoing and not limited to a final report at the end of the year. This makes the reporting 
of evaluation findings to the state-level policy makers more sensitive and precise. Thus, 
the use of an evaluator as program partner is effective at every stage of program 
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development for integrating differing levels of understanding and shifts in accountability. 
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